Typical computing systems include multiple processing units, such as central processing units (CPUs) and graphics processing units (GPUs), which read from and write to a physical memory. The various processing units of a device may implement virtual addressing schemes such that applications being executed by clients can be assigned a contiguous virtual address space without having to set aside a contiguous physical memory space. Each processing unit typically has a memory management unit (MMU) to translate the virtual memory addresses to physical addresses in the physical memory. To perform the required virtual address to physical address mapping, each MMU maintains a separate page table in system memory, and each of these separate page tables can be several megabytes in size.